Regulatory Sandbox Final 
Report: Good With 


A summary of Good With’s participation in the ICO’s Regulatory Sandbox 


Date: April 2023 


Information Commissioner’s Office 


İCO. 


Information Commissioner’s Office 


Contents 

E O O ee mre r NACRE EA AENOR CA ee 3 
2. Product description ..............sseescsscccccccccceeeessssssssnsnseeeeeceececeeeeeesssssnsnaneeeeseeeeceeeeeeeeessseeeees 5 
3. Key data protection Considerations ...............ccscsssssssssccecssccccceceeesssssssssssccceeescccececeseseeees 7 
d: Piding et Cece eire da Sins etrdette cca a KeK IAE EEEIEE ENEI Luts yosiow dale EN EPEAK ENAKI K AN aa KEEA ENRI 21 


Page 2 of 22 


İCO. 


Information Commissioner’s Office 


1. 


1.1 


1.2 


1.3 


1.4 


1.5 


1.6 


Introduction 


The Regulatory Sandbox (‘the Sandbox’) is a service the ICO provides to support organisations that are developing 
products or services which use personal! data in innovative and safe ways, and will deliver a potential public benefit. 


The Sandbox is a free, professional service that is available to organisations of all sizes who meet our entry criteria 
and specified areas of focus. We assess these criteria via our application processes. 


The Sandbox specifically sought projects operating within challenging areas of data protection. Sandbox participants 
have had the opportunity to engage with us, draw upon our expertise and receive our advice on mitigating risks and 
implementing data protection by design and default into their product or service. This helps ensure that appropriate 
protections and safeguards are implemented. 


Good With Limited (‘Good With’) describes itself as a fintech and edtech start-up company seeking to develop mobile 
applications which help educate young adults on personal finance. Good With plans to develop a Financial Virtual 
Assistant (FVA) which will combine various personal data sources to produce a ‘financial readiness score’. Good With 
hopes this readiness score will help to provide the FVA’s users (18-24-year-olds), who might struggle to obtain credit 
via traditional methods, with fairer access to financial products and services. The readiness score will also be informed 
by insights drawn from the user’s interactions with the FVA’s chatbot and their progression through Good With’s 
‘pespoke educational pathway’. Good With will draw these insights by using Artificial Intelligence (AI) to analyse the 
user’s chatbot responses and test scores. It intends to verify the readiness score via accessing the user’s open 
banking data. 


The ICO accepted Good With into the Sandbox on 12 April 2021. The ICO determined that Good With’s project aligned 
with the ICO’s data sharing area of focus at the time of its application. 


The ICO and Good With agreed to work on the following objectives as part of Good With’s bespoke Sandbox plan: 


e Objective one: To complete a granular data mapping exercise to help inform early considerations of data 
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minimisation, data protection by design and default and lawful bases for processing. 


e Objective two: To consider the viability of Good With using cloud providers for voice-to-text operations. This 
objective was rescoped during participation as Good With altered its design to exclude the processing of voice 
data. Following the rescope we agreed to consider controller and processor roles and responsibilities, 
transparency and data sharing. 


e Objective three: To explore ways in which Good With can make use of a privacy dashboard to ensure that 
users are in control of, and fully understand, what personal data of theirs will be processed. 


e Objective four: To consider the intended processing of personal data via the FVA’s AI systems and ensure 
related individual rights are appropriately provided for. This objective also includes themes such as explaining 
decisions made by AI, maximising statistical accuracy and reducing the risk of bias and discrimination. 


e Objective five: Good With to carry out a Data Protection Impact Assessment (DPIA), which will be reviewed by 
the ICO. 


The Sandbox work commenced in August 2021. Good With experienced unexpected resourcing changes during its 
early participation, which resulted in a two month pause to Sandbox participation starting in September 2021. 


The final objective of Good With’s plan was completed during November 2022. This report summarises the work that 
was Carried out during Good With’s time in the Sandbox. 


The ICO has not reviewed a final version of the FVA. This exit report is based on the early, unfinished, designs that 
Good With shared with the ICO and therefore only provides the ICO’s views on the product at that development 
stage. Based on the information it has reviewed, the ICO believes that further work is required from Good With to 
ensure that all the personal data intended to be processed is identified and is compliant with UK data protection 
legislation. Good With has acknowledged that, because the FVA is at such an early conceptual stage, its approaches to 
complying with UK data protection legislation will require further development as it finalises the product design. 
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Product description 


Good With is in the early design phase and prototyping of its FVA. It intends the FVA to be an online, mobile phone 
application which seeks to help its users towards improved financial behaviours, attitudes and awareness. Good With 
anticipates that the FVA will be targeted at users between 18 and 24 years of age. The FVA is not intended to be 
available to users under the age of 18. 


Good With perceives that, via traditional methods of credit scoring, people within this age bracket are often 
disadvantaged or receive unequal access to financial products and services. For example, 18-year-olds often have 
little credit history to use to demonstrate credit worthiness. The FVA’s main function is to produce a financial 
readiness score for its users. Good With intends that score to be used by providers of financial products and services 
when they assess the risk of providing access to their products to these users. Good With believes that this score will 
be a more representative indicator of financial suitability for this age group. 


In pursuing this new approach, Good With intends to make use of, what it terms to be, ‘novel’ indicators of the user's 
financial suitability. This will involve Good With processing various sources of users’ personal data. This includes: 


e insights drawn from the user’s conversational interactions with the FVA’s chatbot. They might include advice 
sought, and subsequent actions taken, related to purchases, budgeting or saving. It might also include the 
user’s attitudes to financial responsibility. The user will initiate these interactions and will interact via text input 
only. An example interaction might be the user seeking advice on the purchase of a particular product. The 
chatbot will then provide advice, such as whether the user should save to buy it outright against the 
affordability of possible finance options. Insights about the user’s financial behaviour may be drawn from the 
user’s responses and actions; 


e the user’s progression through the FVA’s bespoke educational pathway, which includes psychometric testing. 
Good With intends to include psychometric testing which it deems relevant to its user’s financial circumstances. 
For example, those tests will assess topics such as financial wellbeing and perceived risk taking and impulsivity; 
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and 


e the user’s transactional data, provided via open banking access, which will help verify financial insights drawn 
from the above. Good With indicates this data will have the biggest impact on the production of the financial 
readiness score. 


Good With is also seeking to provide the FVA’s users with control over what elements of the service they choose to 
engage with. These elements of the FVA will have varying degrees of impact upon the user’s financial readiness score. 


The various processing activities Good With expects to be carried out by the FVA will include the use of AI or 
automated decision-making. For example, the FVA will respond to and analyse the conversational interactions 
between its chatbot and the user when seeking to detect measures of financial understanding. This information will 
also help determine which elements of the educational pathway are recommended to the user. A further example is 
that the FVA will analyse the user’s transaction history, including categorising them, which will directly contribute 
towards the user’s readiness score. 


Good With states that it also expects to make use of third-party organisations to help operate the FVA. It expects to 
use third parties to support the FVA’s infrastructure and assist with the development and deployment of the AI 
systems. It is also likely to use a Technical Service Provider to supply the open banking data. During its time in the 
Sandbox, Good With had not identified all of the third parties that may process personal data in relation to the FVA. 
As a result, this fell outside of the scope of the Sandbox participation. Where third parties process personal data in 
relation to the FVA controller and processor roles and responsibilities must be appropriately assigned and adhered to. 
Where appropriate, international transfer and data sharing requirements must also be complied with. 


Good With anticipates sharing only the user’s financial readiness score, with providers of financial products and 
services, when the user consents to that data sharing. Those providers might be banks or building societies offering a 
mortgage or loan, a lender offering a credit card or a company offering finance to purchase a product such as a car. 
This is not an exhaustive list. In the current use case, Good With envisage the financial readiness score to be used by 
such providers alongside traditional credit scores provided by other organisations. 
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3.1 


Throughout its Sandbox participation Good With acknowledged that a number of aspects of its FVA are at an early 
stage of development and still require significant development. 


Key data protection considerations 


During its participation within the Sandbox, Good With and the ICO considered a number of key data protection 
themes in relation to the development of the FVA under the objectives set out above. Some of those key areas of 
consideration are outlined below. 


Data minimisation 


3.2 


3.3 


As part of Objective one of the Sandbox plan, the ICO and Good With considered the application of data minimisation 
in relation to the personal data that is expected to be processed by the FVA. The data minimisation principle is set out 
in Article 5(1)(c) of the UK General Data Protection Regulation (UK GDPR). It states that “Personal data shall be 
adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (data 
minimisation)”. Accordingly, as set out in the ICO’s guidance, Good With must ensure that the personal data the FVA 
processes is: 


e adequate - sufficient to properly fulfil Good With’s stated purposes; 
e relevant — it should have a rational link to those purposes; and 
e limited to what is necessary - Good With should not process more personal data than it needs for its purposes. 


The UK GDPR does not define the terms adequate, relevant and limited to what is necessary. These must be assessed 
on a case-by-case basis, taking into account the relevant circumstances. Therefore, the ICO advised Good With that it 
will need to fully understand the context within which the FVA is intended to operate. Good With will also need to 
understand the reasons why personal data will be processed and how this links to its purposes for processing to 
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ensure that it is only processing the personal data it actually needs in that context. This will be important as Good 
With seeks to comply with the data minimisation principle. 


The ICO and Good With worked together to identify and understand Good With’s purposes for processing personal 
data when the FVA is operational. Good With provided an initial outline of its purposes and justifications for 
processing personal data. This provided the ICO with an opportunity to help Good With further develop its approach. 
For example, the ICO advised Good With that it should seek to further understand its purposes for processing at a 
more granular level. These should be kept under regular review during the FVA’s development. The ICO also 
highlighted some additional purposes for processing which Good With had not identified. Those additional purposes 
included, but were not limited to, user identification and authentication, account administration, compliance with 
regulatory requirements and marketing. 


The ICO recommended that Good With should, when its purposes for processing are fully documented, develop 
detailed personal data maps. These personal data maps should include user journeys, and the roles of any third 
parties in processing personal data. Once it has documented its purposes for processing, Good With should then carry 
out a more detailed assessment of its justifications for why the processing of personal data is adequate, relevant and 
limited to what is necessary. To supplement this assessment the ICO recommended that Good With: 


e utilise its own understanding and knowledge of how the FVA works and the credit and financial sectors; 
e assess the impact on the FVA and data subjects if certain items of personal data were not processed; 
e consider whether a less intrusive method of achieving the purpose is possible; and 


e review any relevant sector research or sources related to why, or why not, certain items of personal data (such 
as scores from psychometric tests) are reliable indicators of credit worthiness. This will help Good With assess 
its justifications against the impacts on the user. 


During Objective one, Good With sought to differentiate between what personal data it deemed essential for the use 
of the FVA and what personal data would be optional for the user to provide. The ICO advised that this is important to 
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help ensure Good With does not process excessive amounts of personal data in breach of the data minimisation 
principle. It will also help the data subject retain more control over what personal data is processed. However, Good 
With must also ensure that the personal data it intends to process is adequate and relevant. It must also provide the 
user with clear, and neutral, transparency information. This will help the user make informed choices about the 
provision of personal data in relation to optional elements of the FVA. Good With should also make it clear to the 
FVA’s users if their selection of what personal data they choose to provide may impact the accuracy of their readiness 
score. 


A key area of consideration for this objective was the user’s conversational interaction with the FVA. At first, Good 
With anticipated its users interacting with the FVA by using their voice. The FVA’s chatbot would transcribe the user’s 
voice and respond in text. The interactions would be used for functions such as providing financial advice, generating 
behavioural insights and tailoring the educational pathway to the individual user. Good With told the ICO that the 
voice data would be limited to an audio recording of the user’s speech, which would be transcribed to record what is 
said. It stated it would not create a unique ‘digital voice print’ that could uniquely identify an individual. As a result, 
Good With did not believe this would constitute biometric data. It should be noted that the ICO did not assess this 
conclusion as part of the Sandbox participation. The linked guidance should be used, within the specific context, when 
organisations seek to determine whether the processing of personal data will constitute biometric data. Nevertheless, 
the ICO advised that processing voice data might involve a degree of privacy intrusiveness (depending on how it is 
used) and Good With must be able to justify its use. The ICO and Good With discussed how this processing might 
work and what personal data would be processed for what purposes. 


Good With has since revised its design so that the user will interact with the FVA via text. It felt this approach would 
achieve the same aim and help safeguard the user’s privacy. As a result, the ICO has assisted Good With to minimise 
privacy risk whilst not compromising the development of the FVA. 


The ICO also advised Good With of the importance of complying with the UK GDPR’s storage limitation requirements. 
It must not process personal data longer than it needs to and Good With will need to consider how long it needs to 
retain different items of personal data to achieve its purposes. These timescales should be outlined in a documented 
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retention schedule which is regularly reviewed. Additionally, it will need to implement effective and timely personal 
data removal or deletion processes. Complying with storage limitation requirements should complement Good With’s 
approach to data minimisation. 


Transparency 


3.10 Article 5(1)(a) of the UK GDPR requires that personal data shall be “processed lawfully, fairly and in a transparent 
manner in relation to the data subject (‘lawfulness, fairness and transparency’).” Articles 13 and 14 outline a data 
subject’s right to be informed about how their personal data is processed. These Articles relate to transparency 
requirements when personal data is collected from the data subject and when it has not been obtained from the data 
subject respectively. Within Objectives two and three of the Sandbox plan, the ICO helped Good With consider its 
approach to transparency. 


3.11 The UK GDPR also has specific transparency requirements that relate to automated decision-making, including 
profiling. The ICO and Good With considered these specific requirements during Objective four. That work is 
summarised later in this report. 


3.12 During Sandbox participation, Good With stated its intention to be clear, open and honest in seeking to comply with 
its transparency obligations. This is particularly important where individuals have a choice to provide their personal 
data or not. The ICO advised Good With on how the FVA’s specific use case makes this even more important. For 
example, Good With told the ICO that it intends to make use of novel indicators of financial readiness. It also intends 
to provide individuals with a degree of control over what elements of the FVA they use. It is therefore essential that 
Good With’s approaches to transparency allow individuals to make a sufficiently informed choice about whether to 
provide their personal data, how much of it to provide and the impact this might have on the accuracy of their 
readiness score. Individuals must also be clear about the type of inferences that will be drawn about them from the 
information they provide, and the subsequent impacts of those inferences such as on their ability to obtain finance. 
This will help to mitigate the risk of people’s personal data being processed outside of their reasonable expectations. 
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3.14 


personal data. Good With expects the FVA to make use of a free form chatbot. This means users will be able to enter 
unstructured text data as part of their interactions with the FVA. As the user will have discretion over what personal 
data they will provide to the chatbot, this results in a risk that Good With might unintentionally process personal data 
that it does not want and is not necessary for its purposes. To reduce this risk Good With intends to use an automatic 
redaction procedure which will erase any personal data that is stored but not required. The ICO advised that Good 
With must ensure it provides clear information to users in its privacy notice explaining what information it requires 
users to provide to the chatbot and why. The ICO also advised Good With to: 


e ensure users understand what personal data they should not provide to the chatbot; 


e develop a well-considered and documented rationale related to what personal data will be erased by the 
automatic redaction procedure; and 


e assess how it will comply with the requirements of Article 9 of the UK GDPR if the data subject incidentally 
provides special category data that is processed by Good With. 


The ICO advised that Good With should implement a layered approach to the provision of transparency messaging. 
This involves Good With providing shorter, key transparency messages at important touchpoints during the user 
journey. These messages should be used to supplement a more detailed privacy notice. As it is anticipated that users 
will be able to activate and deactivate some of the FVA’s functions, such as engaging with the chatbot and 
psychometric tests, this approach is important to ensure users in fact retain the level of control over their personal 
data that Good With is seeking to provide. The importance of this approach is further underlined by the fact Good 
With intends to ensure the FVA’s users understand the inferences that may be drawn about them (see section 3.12). 
As a result, Good With intends to supplement its approach to transparency by using: 


e a privacy dashboard; 


e short form privacy notices; 
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e justin time notifications; 
e conversational delivery of transparency information; and 
e video content. 


During Objective three representatives of the ICO and Good With held a workshop to discuss ways in which Good With 
could use a privacy dashboard effectively. Good With told the ICO that its privacy dashboard will be available to users 
at all times in a settings section of the FVA. This will allow the user to toggle on or off any optional functions of the 
FVA and uses of their personal data. 


The discussion at the workshop highlighted a tension between the amount of personal data Good With would like to 
process, and ensuring users are provided with a sufficient degree of control over their personal data. For example, 
Good With believes that the more functions of the FVA the user engages with, and consequently provides personal 
data for, the more beneficial it will be to the user. Good With expects that the FVA will produce the most 
comprehensive understanding of a user’s perceived financial readiness when it has more access to their personal 
data. The ICO advised Good With that the benefits of engaging additional functions of the FVA can, and should, be 
clearly explained to users when they make use of the privacy dashboard. However, for users to remain in control, 
Good With’s messaging should use neutral language to avoid users being ‘nudged’ towards providing more personal 
data than they wish to. The ICO also advised Good With to ensure settings default to privacy first settings to allow 
users to make proactive choices to provide their personal data should they wish. 


The ICO also advised Good With to consider that some of the FVA’s users may be vulnerable, when it drafts its 
transparency wordings. For example, Good With expects that most of its users will be young and that some will be 
financially disadvantaged. Good With should leverage its specific knowledge of the sector it operates in, and its 
audience, to ensure its transparency messaging is comprehensive, understandable to the users it is targeting and free 
from unnecessarily complex terms. 
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3.18 The ICO did not review any finalised wordings of specific transparency messaging and therefore the ICO’s advice was 


provided to Good With based on its conceptual design of the FVA. Good With acknowledged its approaches to 
transparency are under development and will be informed by the ICO’s advice. 


Lawful basis for processing 


3.19 


3.20 


3.21 


To process personal data lawfully a controller must identify an appropriate lawful basis for processing, under Article 6 
of the UK GDPR, for its various processing activities. Where that processing includes special category data an 
additional condition for processing, under Article 9, must also be appropriately identified. Assessing Good With’s 
compliance with these requirements was not a specific area of focus within Good With’s Sandbox plan. However, the 
ICO and Good With found that the various objectives prompted some high-level consideration of these requirements. 
These are briefly summarised within this report and Good With will need to ensure that it has a lawful basis for 
processing personal data. 


During the early phases of its participation Good With intended to rely on consent (under Article 6(1)(a)) and, where 
required, explicit consent (under Article 9(2)(a)) for most of its processing activities. This was informed by Good 
With’s intention to place control in the hands of the user. The ICO advised Good With of the potential difficulties in 
relying on UK GDPR consent across the breadth of the FVA’s processing activities. For example, Good With may 
struggle to demonstrate that consent is freely given and valid for its essential processing activities, if it amounts to a 
precondition of service. The ICO also advised Good With that it should carry out its assessment in a more granular 
way, against each identified processing activity. Whilst consent may be appropriate for some processing activities, 
there may be more appropriate options for others. The ICO highlighted that Good With will need to consider whether 
the training and testing data, used in the development of the FVA’s AI systems, will constitute personal data as part 
of its assessment. Where it does, Good With must identify appropriate options under Article 6 and Article 9 (where 
required). 


An important outcome of this work related to Good With’s privacy dashboard. The ICO informed Good With that just 
because users are able to choose how much personal data the FVA will process this does not necessarily mean 
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consent is appropriate for all of the options provided by the dashboard. If the user engages those options, it will likely 
result in a number of different processing activities being activated. Good With must assess the individual 
circumstances of each processing activity in isolation. It must then decide if it can satisfy one of the lawful bases 
under Article 6 and, where required, one of the conditions under Article 9. 


Following the ICO’s advice, Good With produced an updated consideration of its identified lawful bases as part of its 
DPIA during the latter stages of Sandbox participation. Good With sought to break its assessment down against 
individual purposes for processing and this improved upon the analysis it had provided to the ICO at the outset. 
However, the ICO considers that Good With needs to carry out further work to ensure it has a legal basis for 
processing before the product is launched and any processing of personal data commences. As the FVA continues to 
be developed, Good With must continue to assess all the personal data it intends to process and its purposes for 
processing. It must also detail all of its individual processing activities and purposes, in line with the UK GDPR’s 
requirements on documentation. Good With should then document a thorough assessment of whether it can comply 
with the requirements of Article 6 and (where required) Article 9 and why its identified lawful bases are appropriate. 


AI and automated decision-making 


3.23 


Objective four of Good With’s Sandbox participation focussed on the AI and automated decision-making elements of 
the FVA. Automated decision-making often involves profiling, but it does not have to. Good With will need to consider 
if its intended processing activities amount to the profiling of its users, and assess the risks involved with that 
processing. The ICO and Good With worked together to consider the non-exhaustive key themes identified in our 
guidance on AI and data protection and explaining decisions made with AI (co-badged with The Alan Turing Institute). 
Those themes included: 


e individual rights including rights related to automated decision-making including profiling; 
e explaining decisions made with AI transparently; 


e maximising statistical accuracy; and 
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e reducing the risk of bias and discrimination. 


Given the complexity of AI related processing activities, the below sections of this report summarise some of the key 
points identified in relation to this objective. It does not represent a comprehensive assessment of Good With’s 
planned processing. 


The ICO and Good With worked together to seek to establish whether the requirements of Article 22 of the UK GDPR 
would be engaged. Article 22 states that data subjects “shall have the right not to be subject to a decision based 
solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly 
significantly affects him or her”. This work identified that there can be some challenges in definitively ascertaining 
whether a data subject’s rights, under Article 22, will be engaged by the intended processing at such an early stage of 
a product’s design. The ICO advised Good With to fully assess the role of the FVA, and how it will be used, in 
determining user access to financial products and services such as loans. It must also consider whether there is any 
meaningful human involvement in determining such access and what training needs will be required. Having a clear 
and detailed understanding of the circumstances and impact of the FVA’s deployment is key to Good With ensuring 
the effective exercise of data subjects’ rights and integrating these considerations ‘by design’ early in its product 
development. This will help Good With seek to comply with data protection legislation. Good With must also 
understand these requirements within the differing user journeys of the FVA as data subjects can exercise discretion 
over the elements of the FVA they activate. 


Whilst considering the application of Article 22, the ICO reiterated that Good With must also ensure all individual 
rights are appropriately provided for in its AI systems. In particular, Good With will need to appropriately assess the 
status of its training and testing data. Initially, Good With considered that data to be anonymous on the basis that it 
would be aggregated and direct identifiers would be removed. Anonymous information is not subject to the UK GDPR. 
However, the ICO recommended Good With revisits this assessment. For example, it pointed out that the use of a 
unique identifier, in place of a name or other identifier, might result in the data being pseudonymous, which is still 
considered to be personal data. The ICO also advised Good With that even if the training and testing data lacks 
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associated identifiers or contact details it may still be personal data if an individual can be ‘singled out’, directly or 
indirectly. 


Good With has since informed the ICO that significant work is being carried out to fully understand the status of the 
training and testing data. Where individual rights apply, it intends to provide for them appropriately. It is important 
this work forms part of the FVA’s development to ensure individuals can appropriately exercise the rights they are 
afforded under data protection legislation. 


The UK GDPR, in Articles 13 and 14, requires that specific transparency information is given to data subjects in 
relation to automated decision-making including profiling. Using AI to process personal data can often result in 
complex processing activities involving large volumes of personal data. It can sometimes be challenging to produce 
concise privacy information which adequately details the existence of this type of processing. This information must 
also include the logic involved and the significance and envisaged consequences for individuals. During Objective four, 
Good With explored its intended approaches to providing this specific information. The ICO provided advice to Good 
With on how it can further supplement its transparency approaches summarised earlier in this report. 


The ICO agreed with Good With’s intention to provide a data explanation as part of its transparency messaging. A 
data explanation will help the FVA’s users understand how data about them has been used in making particular AI 
decisions. It also helps individuals challenge such decisions if they believe they are incorrect. The ICO advised Good 
With to further consider whether it can provide the FVA’s users with specific actions they can take to obtain better 
outcomes. However, this should stop short of leaving the FVA vulnerable to being ‘gamed’ inappropriately. The ICO 
considers that providing this type of explanation is important for the following reasons: 


e Good With has determined some of the FVA’s processing activities will engage Article 22, meaning individuals 
will have the right to request human intervention, express their point of view, challenge a decision and obtain 
an explanation about the logic of the decision. 


e Good With hopes to positively influence the financial behaviour of the FVA’s users. 
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e Automated decisions related to the provision of credit are sometimes challenged by data subjects. 


The ICO also considers that it is important for Good With to include a fairness explanation. Good With intends to use 
personal data in a fair, justified and responsible way. Therefore, people should understand why it is reasonable to 
include certain personal data about them when making decisions about their financial readiness. This becomes more 
relevant when taking into account the fact that Good With intends to make use of non-traditional indicators of 
financial readiness such as psychometric test results. For example, individuals may not expect assessments of their 
behaviour to result in scores about their risk-taking or impulsivity that impact decisions about their credit worthiness. 
Providing clear transparency information about this at appropriate times is critical. 


Given its stated commitment to use AI in a justified and responsible way, the ICO advised Good With to also consider 
using a safety and performance explanation. This type of explanation seeks to help individuals understand the steps 
that have been taken to protect them from unjustified outcomes. Taking into consideration the potential impacts of 
the decisions the FVA will make, Good With should inform users by explaining how it will ensure the accuracy, 
reliability, security and robustness of its AI systems. 


Statistical accuracy within an AI system refers to the proportion of answers it gets correct or incorrect. During 
Objective four the ICO helped Good With consider how it will approach seeking to ensure the FVA is statistically 
accurate. The ICO recommended that Good With performs a detailed analysis of the statistical accuracy of the 
financial readiness score that will be assigned to users. This element of the FVA is likely to have the biggest impacts 
on the FVA’s users. Good With should assess where it will set the thresholds that would grant or deny users access to 
financial products and why. The ICO also advised Good With to test how accurately users are assigned to those 
thresholds. For example, Good With must ensure that people who cannot afford a specific loan are not recommended 
for it as the impacts on that individual may result in unjustified adverse outcomes. Similarly, individuals who have 
sufficient means should not be denied access to appropriate financial products. Good With has committed to carrying 
out testing when an early version of the FVA is launched. However, the ICO recommended it ascertains as much 
information as possible about key components of the system before any testing involving personal data commences. 
Good With must also ensure the right to rectification is appropriately provided for to help safeguard against any 
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inaccuracies. Following deployment, Good With should ensure it continues to monitor the FVA’s statistical accuracy 
and takes appropriate action where required. The frequency of its monitoring should be proportionate to the impact 
on users should the FVA produce incorrect outputs. 


The ICO also assisted Good With in considering how it will address risks of bias and discrimination. When AI systems 
are developed they may learn from data which is unbalanced. That could lead to outputs which are biased or 
discriminatory in relation to characteristics such as (but not limited to) ethnicity, gender or health. A key outcome of 
this work identified how ‘proxy variables’ in training data might have a particular relevance to AI systems deployed in 
the credit sector. Proxy variables enable statistical models to reproduce particular patterns of discrimination even 
where they are not intended. For example, the ICO helped Good With to consider how aspects of the training data, 
such as an individual's address or occupation, might reflect traditional assumptions of credit worthiness. This might 
result in biased, discriminatory or unjustified inferences being drawn against other characteristics such as health or 
gender. Good With should use its knowledge of the sector to ensure training data is sufficiently balanced and put in 
place measures to mitigate any areas of risk identified. It should also ensure that reasonable indicators of credit 
worthiness appropriately and proportionately impact the FVA’s outputs such as the financial readiness score. 


As part of the final objective of its Sandbox participation Good With carried out a DPIA. A DPIA must be carried out 
when the processing of personal data is ‘likely to result in a high risk to the rights and freedoms of natural persons’. It 
must be carried out before that processing begins. As part of this objective, the ICO provided iterative, high-level 
feedback on initial drafts to help Good With further develop its DPIA. As Good With has not finished developing the 
FVA, the ICO did not review a DPIA relating to the final product and all of the proposed personal data processing 
activities. Good With should ensure it does not view the completion of its DPIA as a one-off exercise. It should be an 
ongoing process that is subject to regular review. 


Initially, Good With determined that it needs to do a DPIA as the FVA’s intended use of AI and automated decision- 
making constitutes processing likely to result in a high risk. The ICO agreed with that assessment. However, the ICO 
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also advised that Good With’s DPIA should contain more explicit references to why a DPIA is required. This would help 
demonstrate more clearly, in relation to the requirements of Articles 35(3) and 35(4) of the UK GDPR, why Good With 
believes the processing will be high risk. The ICO recommended that Good With considers whether factors such as 
denial of service and large-scale profiling are within the scope of the processing and contribute to the assessment of 
high risk. Having a more granular understanding of the high-risk factors will subsequently help Good With identify and 
seek to mitigate specific risks. 


An important outcome of the DPIA work related to the lawful processing of personal data by the FVA’s automatic 
redaction process (described at section 3.13 of this report) if an individual provides any unwanted and unnecessary 
incidental personal data during interactions with the chatbot. Initially, the ICO consulted with Good With to 
understand this process in more depth. Good With explained that by redaction it means permanent erasure. However, 
it also explained that before any personal data can be permanently erased it must be electronically stored for a short 
period of time. This confirmed that this activity will constitute the processing of personal data. 


Should the user provide any unwanted and unnecessary incidental special category data, Good With found it difficult 
to identify an appropriate Article 9 condition for this processing activity. The ICO provisionally considered that, 
following a high-level review and in specific relation to this use case and context, that it may be possible for Good 
With to rely on explicit consent. However, that is subject to Good With adhering to the UK GDPR’s requirements on 
valid consent and the extra requirements explicit consent entails. For example, Good With would need to put in place 
an appropriate express notice clearly explaining this processing activity (including the process and reason for this 
processing) in detail to the user. The user would then need to expressly consent (opt-in) to the redaction of 
unnecessary special category data by affirming their agreement in a clear statement. Good With will need to analyse 
this further, taking into account how the process is designed, to ensure it identifies the most appropriate Article 9 
condition. The ICO’s view on this is also limited to this specific processing activity, which Good With intends to carry 
out as it is seeks to comply with data minimisation requirements. Should Good With intend to retain or re-process any 
of that personal data for another purpose, such as signposting support services to the FVA’s users, it will need to 
reassess its approach to the requirements of Article 6 and (where required) Article 9 of the UK GDPR. 
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3.38 Relatedly, the ICO advised Good With that it must carefully assess whether the personal data processed during the 
psychometric tests, or inferences it draws from sections of them, might constitute special category data. For example, 
where psychometric testing only seeks to assess the user’s financial requirement to rely on short term loans it may 
not be special category data. However, Good With must assess if the inferences made by psychometric testing, such 
as a user’s perceived level of stress or anxiety, might relate to health or any other special category data. The ICO’s 
guidance on inferences and educated guesses is particularly relevant to this type of processing. Due to the inherently 
sensitive nature of special category data, the result of this assessment should be included in Good With’s ongoing 
consideration of the necessity and proportionality of the processing. 


3.39 Due to the complexity, and potential impact on individuals, of the FVA’s intended processing the ICO considers it is 
appropriate that Good With identified a significant number of AI related risks in its DPIA. Examples of those risks 
include: 


failing to provide adequate transparency information resulting in user’s failing to understand the complexity and 
impacts of the processing; 


inaccurate financial readiness scores resulting in financial hardship if users are approved for loans they cannot 
afford; 


algorithmic bias and discrimination resulting in unjustified adverse outcomes for users, particularly those that 
might be vulnerable; 


users seeking to inappropriately ‘game’ the system to obtain a financial readiness score that is not reflective of 
their circumstances; and 


staff not having sufficient knowledge of how the FVA works to carry out meaningful human reviews where 
Article 22 of the UK GDPR is applicable. 


3.40 The ICO highlighted some additional risks that Good With should also assess in the relevant section of its DPIA. For 
example, are users able to link a joint account to the FVA? If so, what are the risks, impacts, and mitigations for the 
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FVA user and the third party? Good With also describes its approaches to mitigating risks quite broadly in its DPIA. 
Therefore, the ICO advised Good With to refocus on this section of the DPIA. It should assess and ensure that its 
proposed mitigations are focussed and targeted to minimise the impact on data subjects of each identified risk. 


Lastly, Good With stated in its DPIA that some of the identified risks retained a high level of residual risk. It explained 
to the ICO that this was the case, at the time of drafting the DPIA, due to its planned mitigations not yet being 
implemented. Good With will reassess them at the time the mitigations are implemented. The ICO informed Good 
With that after the implementation of mitigations, should any processing which would result in high risks remain, it 
must consult with the ICO before any processing of personal data begins. 


Ending statement 


Good With’s participation in the Sandbox has helped the ICO to further understand some of the challenges 
organisations in the credit sector face when seeking to comply with the requirements of the UK GDPR. In particular, 
the participation has helped both organisations consider data protection requirements alongside novel indicators of 
credit worthiness and complex AI related processing activities. 


Due to the early stage of the FVA’s development, Good With’s participation in the Sandbox has further demonstrated 
how much scope there is to implement data protection by design and default during the conceptual phase of project 
design. It also underlined the importance of developing a full understanding of complex AI processing at an early 
stage. 


By engaging with the Sandbox, Good With has had the opportunity to explore the importance of understanding the 
context within which the FVA will be deployed. This has direct implications on whether individual rights related to 
automated decision-making, including profiling, are engaged. It has also received support and advice in relation to 
key compliance requirements. The ICO and Good With have worked together to focus on some of the risks identified 
by Good With, assess them and mitigate the impact on individuals produced by using AI systems to create complex 
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credit profiles. We also explored how providing effective transparency messages by various means can empower 
individuals to take control of, and understand, how their personal data is processed by AI systems. 


4.4 The ICO believes that Sandbox participation has helped highlight to Good With key compliance requirements that 
require further work. The ICO also hopes that participation has been beneficial to the development of the FVA. Good 
With has acknowledged that it intends to continue working on data protection compliance during the development, 
and after deployment, of the FVA. This provides it with an opportunity to ensure responsible data protection practices 
are central to the FVA’s design. Data protection compliance is not a barrier to innovation. 
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